The impact of agricultural diversification on U.S. crop production: National Corn Models Sensitivity and Robustness Checks
Kate Nelson
12/15/2018
This file contains robustness and sensitivity checks for models evaluating the imapact of agriculutral diversification on U.S. crop production. To reduce the number of models evaluated only national level models for corn using the richness diversity index are examined in this document.
Examining Errors
While model fit statistics and scatter plots of observed and predicted values generally indicate the model fits the data well there are a high number of outliers that can be seen in the predictive p-value plot (0s and 1s). Here we plot the errors (observed yield minus mean predicted yield) in order to identify any trends associated with these outliers. Positive values indicate underestimation of yields and negative values indicate overestimation of yields. As shown below the majority of errors fall near 0. High errors do not appear to occur consistently in the same region over time, but do seem to show highly localized clustering that may be indicative of localized extreme natural disaster events not captured in the model. For example overestimation of yields along small sections of the Mississippi, Ohio, and Missouri rivers and along the eastern coastline can be seen in several years corresponding to years with hurricanes (2011) and low river levels induced by severe drought (2012). These suggest that our model is capturing global and regional variability in annual average trends (particularly climate and weather trends), but does not completely account for short-term (sub-annual) localized extreme weather conditions and other hazards (pest outbreak, wildfire, etc…).
CROSS SECTIONAL MODELS
Individual cross-sectional models are run to illustrate how much the results from the panel model are influenced by differences across space and how variable these results are over time. Results for 2010, 2013, and 2016 are shown below. The cross sectional models suggest that much of the diversity effect does arise from differences that occur across space, but that this is also variable across years. For example, there is no significant effect of diversity in 2013, a year of widespread severe drought, while there is a clear effect of diversity in 2016. Generally, the climate variable functional responses appear to be consistent across years with the effect of precipitation being most variable.
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ------------ ----------- ----------- ------------
(Intercept) 4.6705 0.0241 4.6228 4.6707 4.7174
PERC_IRR 0.0081 0.0008 0.0066 0.0081 0.0095
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 19275.7019 18651.2583 1310.9579 13807.7564 68572.9543
Precision for TP 295.2861 211.9585 68.6953 239.5979 853.1462
Precision for SDD 189.5073 108.3873 61.0438 163.4012 471.0844
Precision for GDD 9.0408 3.1157 4.5224 8.5074 16.6204
Precision for RICH 25229.7911 163647.5896 482.5656 4887.7963 165314.9398
Precision for CNTY 10.9191 0.5765 9.8295 10.9040 12.0968
Phi for CNTY 0.8812 0.0154 0.8483 0.8822 0.9086
Precision for AERCODE.id 18147.6320 18181.7137 1239.3959 12762.1212 66186.6441
Precision for AERCODE.id2 18149.0240 18207.4138 1254.8587 12761.1995 66202.9458
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- ------ ---------
-9916.024 3613.568 3e-07 0.998486[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ----------- ----------- ----------- ------------
(Intercept) 4.7209 0.0243 4.6733 4.7208 4.7686
PERC_IRR 0.0060 0.0006 0.0047 0.0060 0.0073
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 20027.9616 18942.0419 1593.5696 14586.5197 69618.4852
Precision for TP 44.3463 19.8627 16.7663 40.7475 93.0678
Precision for SDD 79.7257 31.9101 34.6633 74.0352 158.0983
Precision for GDD 60.4597 22.6001 27.9507 56.5957 115.4720
Precision for RICH 19704.9624 69437.9931 786.9984 6129.4367 121424.7542
Precision for CNTY 13.3839 0.8447 11.7857 13.3626 15.1106
Phi for CNTY 0.9639 0.0130 0.9328 0.9659 0.9834
Precision for AERCODE.id 18728.3099 18564.2782 1230.6105 13226.9280 67745.6135
Precision for AERCODE.id2 2371.9523 3171.2091 239.0234 1426.4721 10299.2692
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- ------ ---------
-9014.273 3122.628 4e-07 0.997685[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ----------- ----------- ----------- ------------
(Intercept) 4.8295 0.0226 4.7852 4.8295 4.8738
PERC_IRR 0.0049 0.0007 0.0035 0.0049 0.0063
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 18409.4864 18250.5308 1360.5198 13057.4525 66442.6714
Precision for TP 92.9770 77.1663 18.0610 71.3532 297.7739
Precision for SDD 67.6756 31.9810 26.1615 60.8625 148.6798
Precision for GDD 15.0452 5.4372 7.0740 14.1594 28.1797
Precision for RICH 575.6492 379.8899 143.4811 481.5618 1564.3583
Precision for CNTY 17.3350 1.0539 15.3151 17.3206 19.4529
Phi for CNTY 0.7941 0.0234 0.7462 0.7947 0.8382
Precision for AERCODE.id 19387.5176 18707.2829 1434.1851 13942.5922 68564.0110
Precision for AERCODE.id2 32023.1182 27478.1455 4554.4349 24499.2638 104475.2154
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- ------ ----------
-8804.182 3053.256 4e-07 0.9975864[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
Sensitivity to Controls
The next set of three models tests the stability of our results given exclusion or inclusion of the linear controls. If the results are stable we can be assured that any ommitted variables are not correlated with those variables.
MODEL without PERC_IRR
This model removes the percent irrigated area control. The functional response for diversity shifts to show more pronounced effects (greater increasing slope) on yield when the irrigation control is removed. This may indicate that there may be ommitted variables that are related to irrigation, and suggests that incomplete consideration of water access may lead to incorrect attribution of effects to diversity. Note, however that the shift in the response is not statistically significant (the 95% credibility bands overlap with the primary model). In addition, the precipitation response curve shifts slightly for low values of TP (this shift is not statistically significant).
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ----------- -----------
(Intercept) 4.2603 0.0240 4.2130 4.2603 4.3073
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 29.8869 0.4702 28.9167 29.9079 30.7588
Precision for TP 321.7871 156.5944 111.5520 291.4390 710.3334
Precision for SDD 19.5531 5.5115 11.1095 18.7420 32.6049
Precision for GDD 119.4451 48.4787 52.0000 110.4213 239.0363
Precision for RICH 307.8324 131.9028 128.6688 282.0456 635.5308
Precision for CNTY 20.3590 1.0123 18.3681 20.3652 22.3486
Phi for CNTY 0.9991 0.0016 0.9945 0.9998 1.0000
Precision for AERCODE.id 347.8639 83.8819 215.5822 336.6097 543.5932
Precision for AERCODE.id2 10169.7726 2162.1384 6395.5188 10030.4679 14846.4144
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
--------- --------- --------- ---------
-4677.75 2158.998 19425.14 11.37197[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
MODEL without ACRES
This model removes the cultivated acres control.The diversity functional response is not effected by the inclusion/exclusion of the control for agricultural dominance.
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ----------- -----------
(Intercept) 4.1897 0.0221 4.1460 4.1898 4.2330
PERC_IRR 0.0091 0.0006 0.0080 0.0091 0.0103
Precision for the Gaussian observations 29.8919 0.4514 29.0052 29.8918 30.7832
Precision for TP 354.8293 179.4273 124.0014 316.6161 809.9875
Precision for SDD 20.7209 5.5522 11.7782 20.0731 33.4619
Precision for GDD 122.4016 47.1594 48.9967 116.6600 231.1097
Precision for RICH 1157.9762 610.4254 400.5112 1019.4567 2723.3939
Precision for CNTY 21.8925 1.3344 19.0715 21.9978 24.2324
Phi for CNTY 0.9995 0.0010 0.9964 1.0000 1.0000
Precision for AERCODE.id 421.9838 100.7885 246.6001 415.5171 638.2990
Precision for AERCODE.id2 11097.3847 2421.3413 6842.9250 10950.8272 16296.1950
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
--------- --------- --------- ---------
-4742.39 2204.842 19425.11 11.37197[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
MODEL without ACRES and PERC_IRR
This model removes the cultivated acres control and the percent irrigated area control. Results are similar to those seen for the model without PERC_IRR.
[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
FUNCTIONAL RESPONSE CONSISTENCY CHECKS
The next set of results shows the output from very simple models with only the predictor variables of interest to see if we identify the same types of relationships as seen in the full model. The goal here is to determine if the shape of the functional responses in the full model are generally stable and hence likely not the result of confounding or misidentification.
MODEL with only SDI
[[1]]
MODEL with only TP
[[1]]
MODEL with only SDD
[[1]]
MODEL with only GDD
[[1]]
Bayesian Prior Sensitivity Checks
This final set of models examines the sensitivity of the model results to choice of Bayesian prior.
Model with Default Priors
Employs the default, generally uninformative priors assigned by R-INLA. Scale-model is TRUE for all random walk and BYM effects. This model provides results very similar to the primary model reported. Minor differences in the functional response curves for RICH and GDD. Precision estimates for random walk variables slightly higher than in the primary model.
[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ----------- -----------
(Intercept) 4.1765 0.0239 4.1299 4.1764 4.2235
PERC_IRR 0.0084 0.0006 0.0073 0.0084 0.0096
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 30.0115 0.4560 29.1008 30.0189 30.8905
Precision for TP 358.5003 171.1638 122.8435 327.2547 780.9337
Precision for SDD 21.0115 5.9696 12.0962 20.0540 35.2562
Precision for GDD 119.0484 45.5667 50.0529 112.7273 226.1163
Precision for RICH 1080.0021 606.7912 369.6890 931.0314 2657.2037
Precision for CNTY (iid component) 5152.5083 2726.5090 1752.4499 4539.3968 12146.9648
Precision for CNTY (spatial component) 22.5680 1.1755 20.4189 22.5060 25.0383
Precision for AERCODE.id 405.5901 94.1879 249.5245 395.9420 618.2865
Precision for AERCODE.id2 10289.3832 2295.4012 6609.5291 10000.3993 15595.0974
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- -------- ---------
-4764.301 2211.742 19425.1 11.37197Reduced Precision Fixed Effect Prior
Employs the default, generally uninformative priors assigned by R-INLA for random effects. Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given a reduced precision (more uniformative) prior.Results are not significntly differenct from the preceding model and are again consistent with the primary model reported.
[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ---------- -----------
(Intercept) 4.1745 0.0226 4.1301 4.1745 4.2188
PERC_IRR 0.0085 0.0006 0.0073 0.0085 0.0097
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 29.9543 0.4526 29.0616 29.9557 30.8434
Precision for TP 440.3373 202.2692 159.7194 403.7628 937.3097
Precision for SDD 21.4795 5.8026 12.0758 20.8382 34.6499
Precision for GDD 135.0230 51.1584 56.2287 128.3811 253.5859
Precision for RICH 1498.6689 862.0813 478.5554 1290.7853 3738.3003
Precision for CNTY 22.4263 1.1629 20.1179 22.4472 24.6758
Phi for CNTY 0.9993 0.0014 0.9953 0.9999 1.0000
Precision for AERCODE.id 392.6931 92.6895 244.4201 381.0781 606.6409
Precision for AERCODE.id2 10283.1709 2293.0152 6624.9205 9987.7102 15594.7849
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- -------- ---------
-4763.382 2214.372 19425.1 11.37197Penalized Complexity Prior for Spatial Effects
Employs the default settings assigned by R-INLA for a penalized complexity BYM prior. Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given the reduced precision (more uniformative) prior. Results similar to preceding models and consistent with the primary model. Precision for spatial effects slightly higher than in preceding models.
[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ---------- -----------
(Intercept) 4.1735 0.0228 4.1291 4.1734 4.2185
PERC_IRR 0.0085 0.0006 0.0073 0.0085 0.0097
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 29.9599 0.4523 29.0848 29.9552 30.8618
Precision for TP 430.6713 213.0798 158.7570 384.1701 972.8315
Precision for SDD 21.6887 5.8284 12.2329 21.0480 34.9064
Precision for GDD 131.4276 53.7475 58.0876 120.8910 265.4927
Precision for RICH 1468.0367 838.6277 469.3860 1267.5172 3643.4411
Precision for CNTY 22.3931 1.1333 20.2002 22.3848 24.6474
Phi for CNTY 0.9993 0.0013 0.9957 0.9999 1.0000
Precision for AERCODE.id 405.2712 94.1280 251.0549 394.9229 619.5115
Precision for AERCODE.id2 10135.7722 2387.8580 6496.5900 9767.0419 15793.3751
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- -------- ---------
-4764.343 2214.219 19425.1 11.37197Penalized Complexity Prior for Random Walk Effects
Employs the default settings assigned by R-INLA for a random walk (order 1) prior (employs standard deviation of the dependent variable as a scaling factor). Scale-model is TRUE for all random walk and BYM effects. The fixed effect is given the reduced precision (more uniformative) prior.Results are not signficantly different from primary model reported.
[[1]]
[[2]]
[[1]]
[[1]]
[[1]]
[[1]]
[[1]]
Table: Summary Table of Model Estimates
mean sd 0.025quant 0.5quant 0.975quant
---------------------------------------- ----------- ---------- ----------- ----------- -----------
(Intercept) 4.1765 0.0239 4.1299 4.1764 4.2235
PERC_IRR 0.0084 0.0006 0.0073 0.0084 0.0096
ACRES 0.0000 0.0000 0.0000 0.0000 0.0000
Precision for the Gaussian observations 30.0115 0.4560 29.1008 30.0189 30.8905
Precision for TP 358.5003 171.1638 122.8435 327.2547 780.9337
Precision for SDD 21.0115 5.9696 12.0962 20.0540 35.2562
Precision for GDD 119.0484 45.5667 50.0529 112.7273 226.1163
Precision for RICH 1080.0021 606.7912 369.6890 931.0314 2657.2037
Precision for CNTY (iid component) 5152.5083 2726.5090 1752.4499 4539.3968 12146.9648
Precision for CNTY (spatial component) 22.5680 1.1755 20.4189 22.5060 25.0383
Precision for AERCODE.id 405.5901 94.1879 249.5245 395.9420 618.2865
Precision for AERCODE.id2 10289.3832 2295.4012 6609.5291 10000.3993 15595.0974
[[2]]
Table: Model Diagnostic Metrics
DIC CPO MSE R2
---------- --------- -------- ---------
-4764.301 2211.742 19425.1 11.37197